What is Virtual/Augmented Reality?
Virtual reality is a system that allows users to interact with a virtual dimension in real time, typically replicating senses of sight, sound, and touch. When one puts on a head-mounted device, they are shown an entirely different environment. Augmented reality differs from virtual reality in the sense that it only layers information on top of the real world. Many people have likely used augmented reality on their phones, such as with the popular mobile game Pokemon Go.
History of Virtual and Augmented Reality
Virtual reality is a system that allows users to interact with a virtual dimension in real time, typically replicating senses of sight, sound, and touch. When one puts on a head-mounted device, they are shown an entirely different environment. Augmented reality differs from virtual reality in the sense that it only layers information on top of the real world. Many people have likely used augmented reality on their phones, such as with the popular mobile game Pokemon Go.
1935
Virtual and augmented reality both have a surprisingly long history behind them. The first mention of a head-mounted device was as early as the 1930s, with a short fictional story written by Stanley G. Weinbaum called the Pygmalion’s Spectacles. His story is about a man named Dan Berk, who encounters a professor that wants him to try his latest invention: A pair of glasses that allows you to interact with a fictional world. Even though this is only literature, it goes to show that people were thinking about it. Following the story of the Pygmalion’s Spectacles, we do not really see much development in virtual and augmented reality technology until the 60s.
1960s
In 1962, the world’s first virtual reality machine was unveiled, called the Sensorama, designed by Morton Heilig. Heilig is generally coined as the father of virtual reality, since he did create the first virtual reality machine after all. The Sensorama allowed you to observe a film as if you were actually there. The machine would play a three-dimensional film in a first-person perspective. On top of this the machine replicated the senses of touch, sound, and even smell. A handlebar that would vibrate, speakers to hear the conversations, and fans to spread the aroma that you were supposedly in. While this is an intriguing piece of technology, it falls short of great potential, since you only are allowed to observe the environment, not interact with it. Six years later, we see the first augmented reality head-mounted device, named the Sword of Damocles, made by computer scientist Ivan Sutherland and his student Bob Sproul. It had to be mounted to the ceiling due to it being too heavy for the average user. This headset was connected to a computer, and overlaid a wireframe cube over the real world.
1980s
The 80s is when we start to see interaction in the virtual worlds. In 1985, VPL created the DataGlove, which allows you to hold a virtual item in the real world. The glove tracked the user’s movements using fiber optic cables and magnetic sensors. VPL then went on to partner up with NASA in 1987 to create the DataSuit, the first commercial virtual reality headset. The main drawback with this setup, is that it was far too expensive for the average consumer to purchase. The headset, combined with the DataGlove and the computers needed to run the software, was $250,000. With the price tag being this hefty, VPL called it quits and filed for bankruptcy in 1990.
1990s
During the 90s, we saw some trial and error regarding virtual reality. In 1987, Johnathan Waldern established W-Industries, in which they developed a variety of virtual reality headset prototypes. A few years later, they presented their virtual reality arcade, model number 1000SU.
In 1991, Virtuality unveiled the 1000CS, CS standing for Cyber Space and 1000SD, SD standing for Sit Down, virtual reality arcades. The 1000SD allowed you to fly a virtual jet, and even had a motion-rig setup to simulate the movements of the jet. The 1000CS on the other hand, had you standing on a platform surrounded by a large ring holding a pistol-shaped controller. The system also used a magnetic field to track the user’s movement. From 1993 to 1995, Nintendo, Sega, and Atari attempted to all make their own virtual reality headsets, which all failed. Sega along with Mark Pesce’s company, Ono-Sendai, created an affordable headset. When the headset came out, it failedflopped due to it being too immersive and realistic, leading to users running around and injuring themselves. On top of this, some users experienced motion sickness while wearing the headset. Following Sega’s headset was Atari’s. Atari paired up with both Virtuality and Jaguar to create a headset that had better tracking, improved resolution, and a much more stable connection. Virtuality kept on investing in the development of the headset, but when Atari merged with JTS, the entire project was canned.
2010s
From 2010 and beyond, we start to see some serious improvement within the virtual and augmented reality fields. Palmer Luckey is a notable name in the virtual reality industry due to actually releasing the first commercially affordable headset. Luckey was not very dissatisfied with the ways that virtual reality was being implemented has been implemented, so he collected some old headsets and learned about the technologies used in them to make his own improvements. He enrolled at the University of Southern California’s Institute for Creative Technologies to develop his own headsets. Luckey reached out to John Carmack, creator of Doom and Wolfenstein, to work together on the headsets. By the time E3 2012 took place, Carmack showcased a modified version of Doom 3 using Luckey’s headset prototype. Luckey then goes on to found Oculus VR along with some colleagues. They released the Development Kit 1, Shortly after in 2014, Facebook purchased Oculus, and released the Development Kit 2. These kits were sent out to developers to create games in virtual reality. Once the games were ready and polished, the headset officially released as the Oculus Rift in 2016 at $600. While this price is much better than any other headset that came out before, users still needed a computer capable of running the software to use the headsets. Most users would be looking at around $1000 to build a desktop that could play games designed with these headsets. This is still much less compared to the over $250,000 you had to spend on NASA’s DataSuit setup.
In 2015, Taiwanese phone company HTC and Valve, creator of the largest gaming platform Steam, announced Vive, a virtual reality headset that would use Valve’s external tracking method called the Lighthouse. These Lighthouses would use infrared tracking to track the user’s head and hand positions, as well as showing the users their boundaries. The HTC Vive released in 2016 for $800 as well as the Oculus Rift mentioned before. On top of the two releases from HTC and Oculus, Sony also released their PSVR. While these headsets offered a better experience than anything before them, as well as being more affordable,. you still needed a computer that could run it as mentioned before. This is where smartphone virtual reality systems showed up. Since almost everyone has a smartphone in their pockets, all they would need is a headset such as a Google Cardboard or a Samsung Gear VR to have a decent experience. I’ve had the chance to try out the Samsung Gear VR with my Galaxy S6 at the time, and honestly it was nothing special. Having no controllers made the experience awkward as you had to keep your right hand near the headset to select any options. It was cool being able to watch videos on a huge screen though. Smartphone virtual reality died out quickly since it was inconvenient for many users including myself.
In 2019, Oculus released two new headsets, the Rift S which still needed a desktop, and the standalone Quest. The Quest was a game changer since you did not need a desktop to run it, and for $400 it was a fairly good experience. Users also had the ability to hook up the headset to a computer using a Quest Link cable to still get a traditional desktop virtual reality experience. Along with the two new entries from Oculus, Valve and HTC split up to make their own headsets. Valve released the Index which costs $1000, and HTC released the Vive Cosmos which costs $700. I’ve also had the pleasure of being able to own a Valve Index, and let me tell you that it truly is an immersive experience. Their controllers, called knuckles, have independent finger tracking. Games that utilized this mechanic led to a much more engaging experience. For example, in the popular socializing platform VR Chat, I was able to give people a thumbs up or a high five depending on the avatar I was using. It added a layer of immersion that was not as well thought out as before.
2020s
Even though this decade has only started 3 years ago, there is a lot that has been released. For starters, Facebook starts focusing on their standalone headsets, halting production of the Rift S. They soon released the Quest 2, a suitable upgrade over the original Quest, and at only $300, $100 less than its predecessor . Valve released Half Life: Alyx on Steam. This game was full of details, which characters that felt real when standing next to them. I received a copy of this game with my Index headset, and I must say that it was a breathtaking experience. StarVR and Varjo released some high-end headsets mainly targeted towards business. The StarVR had built in eye tracking, which once again leads to a much more immersive experience depending on the application being used. HP also releases the Reverb G2, costing $599 and has the clearest resolution compared to the other headsets within this time. Facebook, now called Meta, released their Meta Quest Pro headset in October 2022, for a steep price of $1500. This headset is designed to be used as a mixed reality work headset. The passthrough mode allows you to see the real world, with some extra screens layered on to have a much more productive environment to work with. Aside from the passthrough mode, the headset can also be used as a virtual reality headset to attend virtual meetings. The headset even has some facial tracking features, so that your avatar can mimic your expressions during these meetings.
Last but not least Sony released the PSVR 2 this past February for $550, offering a great experience at a competitive price. This headset combined with a Playstation 5 will only net you $1100, and has a lot of features to boast. The only drawback as of now is that there is a limited selection of games available for the PSVR when compared to the vast library available on PC and Meta. The only thing left for this field are just rumors of HTC, Apple, Valve and many other companies teasing their new potential headsets. This finishes up the semi-summarized history of virtual reality as well as some augmented reality.
Hardware and Software Components
The two main components of a virtual or augmented reality system boils down to both hardware and software. Let’s describe a typical computer-powered setup. On the hardware side, users will typically wear a head-mounted device that contains sensors to track the head position. Depending on the headset, it may include eye or face tracking. The headset will generally also have integrated speakers, or have an option to plug your own headphones in. Following the headset are the controllers. The controllers will track your hand position in the virtual environment, and some controllers could also include independent finger tracking. Depending on the setup, you could also have the lighthouses, which emit infrared to add additional tracking to the headset and controllers, as well as defining your boundary. All of these components are hooked up to the desktop, which will render the environment as well as updating your position. Rendering is a process in which a computer generates a 2- or 3-dimensional image from a program that contains code. This is not the norm for setups, as we’ve seen Meta’s standalone options giving the full experience without the need for a desktop.
Along with virtual reality there is also augmented reality, which layers information over the real world. While the Meta Quest Pro has augmented reality components, the Microsoft HoloLens is a more traditional augmented reality headset. You see the real world as it is, but you also have a screen or two on the side for video calls. The information can even be layered over anything you are working on. An example of this is with medical care as surgeons can see information about the patient they are operating on.
Uses of Virtual and Augmented Reality
Virtual and augmented reality have a multitude of uses. For the most part, it is used for entertainment. Aside from this though, businesses can analyze data in an efficient manner using multiple screens instead of being limited to 2 or 3.
Big data can be easily digestible using augmented reality, providing visualizations that make things clearer for the user. Artificial intelligence has been a major talking point recently with chatbots such as chat-gpt being able to write code for users and answer questions when prompted. Artificial intelligence can be implemented into virtual and augmented reality systems. For augmented reality, users would be able to use AI for speech recognition, putting real time subtitles for people in foreign countries. For virtual reality, AI can be used to train military pilots using flight simulators, with the AI being the opponents replicating real pilots. There are truly endless possibilities.
Conclusion
Virtual and augmented reality systems are a tool that can be used for a majority of applications. They have come a long way, starting from just a conceptual story, to head-mounted devices that can enhance daily lives. Visualizing big data has never been easier, and AI provides good immersion when with non-player characters. Both systems will only improve over time, to the point where it is accessible to everyone.
Jonathan Pierre-Louise
Author